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FOREWORD 


In  April  2002,  the  Army  Training  and  Leader  Development  Panel  (ATLDP)  released  the 
results  of  its  survey  of  35,000  Noncommissioned  Officers  (NCOs).  The  ATLDP’s 
recommendations  included  the  need  for  regular  assessment  of  Soldiers’  technical,  tactical,  and 
leadership  skills.  The  need  for  regular  assessment  of  Soldiers  coincides  with  the  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences’  (ARI)  research  program  on  NCO 
development  and  assessment.  ARI’ s  research  program  began  with  Soldier  Characteristics  of  the 
21st  Century  (Soldier21 )  to  identify  potential  knowledges,  skills,  and  attributes  (KSAs)  for  future 
Soldiers  and  continued  with  Maximizing  list  Century  Noncommissioned  Officers  Performance 
(NC021 )  to  identify  and  validate  potential  indicators  of  the  KSAs  for  use  in  junior  NCO 
promotion.  The  Performance  Measures  for  21st  Century  Soldier  Assessment  (PerformMU) 
extends  the  research  program  with  a  three-phase  effort  to  examine  the  feasibility  of 
comprehensive  competency  assessment.  The  first  phase  is  an  investigation  of  the  issues  and 
possible  resolutions  for  development  of  a  viable  Army-wide  program  including  the 
Demonstration  Competency  Assessment  Program  (DCAP),  which  is  a  prototype  for  Army-wide 
competency  assessment.  The  second  phase  extends  the  feasibility  investigation  through 
development  of  five  Military  Occupational  Specialties  (MOS)  competency  assessments  as  well 
as  a  self-assessment  and  development  module  to  accompany  the  DCAP.  The  third  phase  is  an 
analysis  of  the  prototype  program  to  provide  recommendations  on  feasibility,  resource 
requirements,  and  implementation  strategies  for  competency  assessment.  This  multi-volume 
report  documents  activities  supporting  the  first  goal  of  Phase  I — issues  impacting  overall 
recommendations  for  Army-wide  assessment — and  also  describes  the  development  of  the  DCAP 
assessment.  The  prototype  DCAP  assessment  and  elements  of  the  recommended  delivery  system 
will  be  pilot  tested  in  Phase  II  of  the  project.  Program  design  issues  identified  here  will  inform 
future  deliberations  about  the  design,  implementation,  and  maintenance  of  an  operational 
assessment  program. 

The  research  presented  in  this  report  has  been  briefed  to  the  Deputy  Chief  of  Staff,  G-l, 
on  8  Oct  2003  and  the  Chief  of  Enlisted  Professional  Development,  Directorate  of  Military 
Personnel  Policy  on  13  Nov  2003.  It  was  briefed  to  the  Sergeant  Major  of  the  Army  on  28  Jan 
2003  and  30  Mar  2004.  It  has  been  periodically  briefed  to  senior  NCO  representatives  from  U.S. 
Army  Training  and  Doctrine  Command  (TRADOC),  Office  of  the  G-l,  U.S.  Army  Forces 
Command  (FORSCOM),  U.S.  Army  Reserve  (USAR),  and  the  Army  National  Guard  (ARNG) 
as  members  of  the  Army  Testing  Program  Advisory  Team  (ATP AT). 


The  goal  of  ARI’s  Selection  and  Assignment  Research  Unit  is  to  conduct  research, 
studies,  and  analysis  on  the  measurement  of  attributes  and  performance  of  individuals  to  improve 
the  Army’s  selection  and  classification,  promotion,  andje^ssignment  of  officers  and  enlisted 
Soldiers. 


PAUL  A.  GADE 


Acting  Technical  Director 
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ARMY  ENLISTED  PERSONNEL  COMPETENCY  ASSESSMENT  PROGRAM  PHASE  I 
(VOLUME  II):  DEMONSTRATION  COMPETENCY  ASSESSMENT  PROGRAM 
DEVELOPMENT  REPORT 


Executive  Summary 


Research  Requirement: 

The  Army  is  changing  to  meet  the  needs  of  the  21st  century.  Soldiers  at  all  levels  must 
possess  the  interpersonal,  technical,  and  organizational  knowledge,  skills,  and  other  attributes  to 
perform  effectively  in  complex  technical,  information-rich  environments,  under  multiple  and 
changing  mission  requirements,  and  in  semi -autonomous,  widely  dispersed  teams.  The  Army 
needs  an  integrated  Soldier  assessment  system  to  support  these  demands. 

The  need  for  Soldier  assessment  is  most  acute  at  the  time  of  promotion  into  the 
Noncommissioned  Officer  (NCO)  ranks.  It  is  at  this  juncture  that  job  competency  merges  with 
leadership  and  supervisory  requirements  and  there  are  distinct  changes  in  the  concept  of 
Soldiering.  In  June  2000,  the  Chief  of  Staff  of  the  Army  established  the  Army  Training  and 
Leader  Development  Panel  (ATLDP)  to  chart  the  future  needs  and  requirements  of  the  NCO 
corps.  After  a  2-year  study  which  incorporated  the  input  of  35,000  NCOs  and  leaders,  a  major 
conclusion  and  recommendation  was:  “ Develop  and  sustain  a  competency  assessment  program 
for  evaluating  Soldiers’  technical  and  tactical  proficiency  in  the  military  occupational  specialty 
(MOS)  and  leadership  skills  for  their  rank”  (Department  of  the  Army,  2002). 

To  meet  the  Army’s  need  for  job-based  measures,  the  U.S.  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences  (ARI)  instituted  a  3-year  program  of  feasibility  research  to 
identify  a  viable  approach  for  development  of  a  Soldier  assessment  system  that  is  both  effective 
and  affordable.  HumRRO  is  the  prime  contractor  for  this  project,  along  with  Job  Performance 
Systems,  Inc.  (JPS).  The  ARI  program  (called  PerformM21)  has  three  phases: 

■  Phase  I:  Feasibility  and  Alternative  Designs 

■  Phase  II:  Design  Selection  and  Prototype  Measure  Development  and  Testing 

■  Phase  HI:  Performance  Measure  Evaluation  and  System  Recommendations 

This  report  focuses  on  the  procedures,  methods,  and  products  involved  in  the 
development  of  a  prototype  assessment  measure.  As  this  project  evolved,  it  became  known  as  the 
Demonstration  Competency  Assessment  Program  (DCAP). 

Several  significant  events  within  the  Army  coincided  with  ARI’s  efforts  in  this  area.  The 
ATLDP  recommendation  resulted  in  the  Office  of  the  Sergeant  Major  of  the  Army  (SMA)  and 
the  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  initiating  a  series  of  reviews  and 
consensus  meetings  with  the  purpose  of  instituting  a  Soldier  competency  assessment  test. 
Ongoing  efforts  within  the  Army  G-l  to  revise  the  semi-centralized  promotion  system  (which 
promotes  Soldiers  to  the  grades  of  E5  and  E6)  also  were  investigating  the  use  of  performance 
(test)  based  measures  to  supplement  the  administrative  criteria  used  to  determine  promotion. 
Ultimately,  the  three  interests  (ARI,  SMA/TRADOC,  and  G-l)  coalesced  and  ARI  sought  to 
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incorporate  the  program  goals  and  operational  concerns  of  all  Army  stakeholders,  while  still 
operating  within  its  research-mandated  orientation. 

Procedure: 

The  fusion  of  the  ARI  project  with  the  efforts  of  the  SMA  and  TRADOC  served  to 
provide  the  demonstration  test  some  specific  operational  parameters,  which  allowed  a  more 
focused  developmental  effort.  We  determined  that  the  operational  goal  would  be  to  have  a 
knowledge  or  knowledge-based  test  of  approximately  150  items  that  could  be  administered 
within  about  a  2-hour  time  frame  via  a  web-connected  computer.  The  test  would  be  initially 
designed  to  be  administered  to  E4  Soldiers  being  considered  for  promotion  to  E5  pay  grade 
(Sergeant).  A  single  test,  suitable  for  all  Soldiers,  regardless  of  MOS  and  regardless  of 
component  (Active  Army,  United  States  Army  Reserve,  Army  National  Guard)  would  be 
•developed. 

Early  in  the  project  we  constituted  a  group  of  senior  NCOs  -  the  Army  Test  Program 
Advisory  Team  (ATP AT)  -  to  advise  on  the  operational  implications  of  Army  assessment  testing. 
The  ATP  AT  serves  two  distinct  purposes.  First,  it  provides  input  for  the  needs  analysis 
requirements  of  the  project,  primarily  by  providing  insight  into  operational  implications  and  real- 
world  feasibility  of  the  program.  Second,  it  serves  as  the  oversight  group  for  development  of  the 
DCAP  as  well  as  a  resource  in  identifying  and  developing  content  for  the  test.  Additionally,  the 
ATPAT  is  a  working  group  that  provides  product  reviews,  subject  matter  expertise,  and,  as 
needed,  assistance  in  the  process  of  developing  prototype  instruments  and  trial  procedures.  An 
additional  benefit  of  the  ATPAT  is  to  serve  as  a  conduit  to  explain  and  promote  the  PerformM21 
project  to  various  Army  agencies  and  constituencies. 

The  ATPAT  met  three  times  in  2003,  providing  guidance 
about  (a)  whether  and  how  the  assessment  will  be  used  in  personnel- 
related  areas;  (b)  the  steps  and  organizational  implications  to 
implementing,  maintaining,  and  growing  an  Army  test;  (c)  identifying 
the  considerations  that  must  be  taken  into  account  to  operationalize  an 
Army-wide  testing  program;  and  (d)  how  the  Army-wide  test  will  fit 
with  other  programs  and  activities  such  as  self-development,  unit 
training,  NCO  Education  System  (NCOES),  deployments,  NCO 
Evaluation  Record  (NCOER)  system,  TDA  staffing,  transition, 

Soldier  tracking  and  assignment,  Future  Force,  and  training 
publications  and  updates.  In  addition,  the  group  discussed  and 
provided  expertise  related  to  the  determination  and  specification  of 
the  content  domains  of  the  DCAP.  They  have  also  helped  to  identify 
resources  for  test  item  development  and  field  testing. 

For  the  most  part,  the  DCAP  development  followed  standard 
instrument  development  steps,  which  are  outlined  in  the  figure  shown 
to  the  right.  Due  to  time  and  resource  constraints,  we  were  not  able  to 
conduct  a  typical  job  analysis  for  the  prototype  DCAP.  We  relied 
instead  on  our  collective  experience  with  the  Army  and  guidance  from  the  ATPAT.  This  input  was 
used  to  develop  a  test  blueprint,  which  is  a  map  of  the  content  to  be  tested.  The  careful 


x 


development  of  a  blueprint  for  the  content  areas,  subject  areas,  and  tasks  helps  to  ensure  that  the 
DCAP  will  cover  the  information  that  is  important  for  Soldiers  to  know  if  they  are  to  be  promoted. 

Following  blueprint  development,  HumRRO  staff  developed  items  and  adapted  items 
from  previous  research  projects  (i.e.,  Select21  and  Project  A)  (J.  Campbell  &  Knapp,  2001; 
Knapp,  2003).  Subject  matter  experts  (SMEs)  from  the  Army  and  within  HumRRO  reviewed 
these  items.  Reviews  by  subject  matter  experts  (e.g.,  instructors,  the  ATP  AT)  ensures  that  the 
items  are  clearly  written,  accurately  key  correct  and  incorrect  responses,  are  verifiable,  and  cover 
appropriate  topics. 

Findings: 

The  ATP  AT  members  reviewed  the  draft  content  areas  and  assigned  them  weights,  which 
determined  what  proportion  of  the  test  would  be  allocated  to  each  of  four  content  areas.  The 
result  of  this  exercise  was  that  Common  Tasks  Skill  Level  One  (SL1)  would  comprise  46.2%  of 
the  test  while  Skill  Level  Two  (SL2)  Common  Tasks  would  cover  13.5%;  History/Army  Values 
would  encompass  15.4%;  Leadership  13.1%;  and  Training  would  constitute  9.11%  of  the  test. 
The  ATP  AT  also  rated  the  criticality  of  subjects  within  the  content  areas. 

An  additional  part  of  the  promotion  assessment  will  be  the  administration  of  a  situational 
judgment  test  that  was  developed  for  the  NC021  project  (Ford  et  al.,  2000;  Knapp  et  al.,  2002)  - 
the  Leadership  Judgment  Exercise  (LeadEx).  The  LeadEx  was  shown  to  be  predictive  of  success 
at  the  E5  and  E6  pay  grades;  that  is,  Soldiers  who  performed  well  on  the  LeadEx  also  were 
highly  rated  on  assessments  performed  by  their  supervisors.  The  LeadEx  assesses  eight 
performance  dimensions:  (1)  Problem  solving  and  decision  making  skill;  (2)  Motivating,  leading, 
and  supporting  subordinates;  (3)  Directing,  monitoring,  and  supervising  work;  (4)  Training 
others;  (5)  Relating  to  and  supporting  peers;  (6)  Team  leadership;  (7)  Concern  for  Soldier  quality 
of  life;  and  (8)  Cultural  tolerance. 

Utilization  of  Findings: 

The  pilot  DCAP  will  be  administered  via  the  Internet  to  Soldiers  at  proctored  sites  in  the 
continental  United  States  (CONUS)  and  overseas  (OCONUS).  After  comparing  many  systems, 
Questionmark’s  Perception®  software  was  selected  as  the  best  product  on  the  market  to  meet  the 
program’s  needs,  both  in  instrument  development  and  for  computer  based  delivery. 
Administration  will  be  through  the  Army’s  Digital  Training  Facilities  (DTF),  which  are  part  of 
the  Army’s  existing  computer  based  program  designed  to  deliver  the  Army’s  distributed  learning 
training  program. 

During  Phase  II  of  PerformM21,  the  DCAP  core  assessment  will  be  pilot  tested.  The  pilot 
test  is  slated  to  begin  in  early  2004  and  will  target  administration  to  between  600  and  1000 
Soldiers  in  the  Active  Army,  Army  National  Guard,  and  United  States  Army  Reserve. 

We  will  develop  and  distribute  preparation  materials  that  Soldiers  can  use  as  a  guide  in 
preparing  for  the  pilot  test.  This  guide  will  provide  information  about  the  assessment  program, 
the  content  areas  of  the  assessment,  preparation  strategies,  and  references  for  the  manuals  used  in 
development  of  the  test  content. 
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Planned  analysis  includes  item  statistics  (e.g.,  percent  correct,  point-bi serial  correlations) 
for  each  item.  Items  exhibiting  poor  item  statistics  will  be  flagged  for  review  and  modification  or 
deletion.  We  will  also  analyze  the  data  to  determine  whether  there  are  differences  in  scores 
associated  with  gender,  race/ethnicity,  rank,  or  service  component  (USAR,  ARNG,  Active 
Army).  Depending  on  distribution  of  the  pilot  test  sample,  we  will  also  try  to  analyze  results 
based  on  Army  jobs.  Most  likely,  we  will  collapse  jobs  into  combat,  combat  support,  and  combat 
service  support  classifications.  We  will  compute  reliability  estimates  for  the  entire  assessment,  as 
well  as  for  sections  of  the  instrument  and  examine  the  correlation  between  scores  on  the  four  test 
sections,  particularly  Leadership  and  LeadEx  scores. 
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ARMY  ENLISTED  PERSONNEL  COMPETENCY  ASSESSMENT  PROGRAM  PHASE  I 
(VOLUME  II):  DEMONSTRATION  COMPETENCY  ASSESSMENT  PROGRAM 

DEVELOPMENT  REPORT 


Roy  C.  Campbell,  Patricia  A.  Keenan,  Karen  O.  Moriarty, 

Deirdre  J.  Knapp,  and  Tonia  S.  Heffner 

Introduction  and  Background 

The  Army  is  changing  to  meet  the  needs  of  the  21st  century.  Soldiers  at  all  levels  must 
possess  the  interpersonal,  technical,  and  organizational  knowledge,  skills,  and  other  attributes  to 
perform  effectively  in  complex  technical,  information-rich  environments,  under  multiple  and 
changing  mission  requirements,  and  in  semi-autonomous,  widely  dispersed  teams.  The  Army 
needs  an  integrated  Soldier  assessment  system  to  support  these  demands. 

The  need  for  Soldier  assessment  is  most  acute  at  the  time  of  promotion  into  the 
Noncommissioned  Officer  (NCO)  ranks.  It  is  at  this  juncture  that  job  competency  merges  with 
leadership  and  supervisory  requirements  and  there  are  distinct  changes  in  the  concept  of 
Soldiering.  In  June  2000,  the  Chief  of  Staff  of  the  Army  established  the  Army  Training  and 
Leader  Development  Panel  (ATLDP)  to  chart  the  future  needs  and  requirements  of  the  NCO 
corps.  After  a  2-year  study  which  incorporated  the  input  of  35,000  NCOs  and  leaders,  a  major 
conclusion  and  recommendation  was:  “ Develop  and  sustain  a  competency  assessment  program 
for  evaluating  Soldiers’  technical  and  tactical  proficiency  in  the  military  occupational  specialty 
(MOS)  and  leadership  skills  for  their  rank”  (Department  of  the  Army,  2002). 

The  Army  does  not  currently  have  an  objective  competency  assessment  test  as  part  of  its 
promotion  system.  In  the  early  1990s,  the  Army  abandoned  its  Skill  Qualification  Test  (SQT) 
program  due  primarily  to  maintenance,  development,  and  administration  costs.  Cancellation  of 
the  SQT  program  left  a  void  in  the  Army’s  capabilities  for  assessing  and  forecasting  job 
performance  qualification.  Re-instituting  a  new  performance  assessment  system  must  address  the 
factors  that  forced  abandonment  of  the  SQT.  Since  then,  technological  advances  have  occurred 
that  can  reduce  the  developmental  and  administrative  burdens  encountered  with  SQT  and  will 
play  a  critical  role  in  a  new  performance  assessment  system. 

Purpose  of  the  Report 

To  meet  the  Army’s  need  for  job-based  measures,  the  U.S.  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences  (ARI)  instituted  a  3-year  program  of  feasibility  research  to 
identify  a  viable  approach  for  development  of  a  Soldier  assessment  system  that  is  both  effective 
and  affordable.  HumRRO  is  the  prime  contractor  for  this  project,  along  with  Job  Performance 
Systems,  Inc.  (JPS).  The  ARI  program  (called  PerformM21)  has  three  phases: 

■  Phase  I:  Feasibility  and  Alternative  Designs 

■  Phase  II:  Design  Selection  and  Prototype  Measure  Development  and  Testing 

■  Phase  HI:  Performance  Measure  Evaluation  and  System  Recommendation 
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Phase  I  of  the  program  (which  corresponds  roughly  to  year  one  of  the  3-year  overall 
effort)  had  three  primary  goals: 

■  Goal  1 :  Determine  the  feasibility  and  inherent  trade-offs  in  the  development  of  an 
operational  and  affordable  individual  performance  assessment  system  for  Army 
enlisted  Soldiers. 

■  Goal  2:  Identify  the  major  design  considerations  and  elements  of  such  a  system. 

■  Goal  3:  Develop  a  prototype  assessment  measure. 

Although  all  three  goals  are  interrelated,  this  report  will  focus  on  the  procedures, 
methods,  and  products  involved  in  meeting  Goal  3:  The  development  of  a  prototype  assessment 
measure.  As  the  product  of  this  goal  evolved,  it  became  known  as  the  Demonstration 
Competency  Assessment  Program  (DCAP). 

This  report  describes  the  major  steps  performed  and  products  that  were  involved  to 
produce  a  prototype  test.  Where  appropriate,  it  includes  the  procedures  that  were  involved; 
however,  it  is  not  a  chronology  of  the  development,  nor  does  it  document  all  the  decision  points. 
The  development  of  the  DCAP  is  presented  in  the  following  sections. 

■  Sources  of  Guidance  and  Input 

■  Overview  of  DCAP  Development  Process 

■  Army-wide  Requirements  Analysis 

■  Blueprint  Development 

■  Item  Development  and  Review 

■  Plan  for  Pilot  Testing 

■  Conclusions 


Sources  of  Guidance  and  Input 


Sergeant  Major  of  the  Army 

The  impetus  to  include  individual  Soldier  assessment  research  in  ARI’s  programmed 
requirements  began  prior  to  2000  and  was  based  on  a  number  of  considerations  regarding  trends 
and  requirements  in  Soldier  selection,  classification,  and  qualifications.  Concurrently,  there  were 
several  significant  events  within  the  Army  that  coincided  with  ARI’s  efforts  in  this  area.  The 
aforementioned  ATLDP  recommendation  resulted  in  the  Office  of  the  Sergeant  Major  of  the 
Army  (SMA)  and  the  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  initiating  a  series 
of  reviews  and  consensus  meetings  with  the  purpose  of  instituting  a  Soldier  competency 
assessment  test.  Ongoing  efforts  within  the  Army  G-l  to  revise  the  semi -centralized  promotion 
system  (which  promotes  Soldiers  to  the  grades  of  E5  and  E6)  also  were  investigating  the  use  of 
performance  (test)  based  measures  to  supplement  the  administrative  criteria  used  to  determine 
promotion.  Ultimately,  the  three  interests  (ARI,  SMA/TRADOC,  and  G-l)  coalesced  and  ARI 
sought  to  incorporate  the  program  goals  and  operational  concerns  of  all  Army  stakeholders, 
while  still  operating  within  its  research-mandated  orientation. 

The  fusion  of  the  ARI  project  with  the  efforts  of  the  SMA  and  TRADOC  caused  some 
alteration  in  the  original  goals  of  the  ARI  demonstration  test.  However  it  also  served  to  provide 
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the  demonstration  test  some  specific  operational  parameters,  which  allowed  a  more  focused 
developmental  effort.  Working  through  TRADOC  and  utilizing  primarily  Command  Sergeant 
Major  channels  and  resources,  the  SMA  issued  the  following  implementation  guidance  and  goals 
(Heffner,  2003): 

■  The  test  is  to  be  used  for  promotion  purposes  with  direct  promotion  application  for 
the  semi -centralized  (E5,  E6)  promotion  system. 

■  The  test  and  its  supporting  system  must  serve  the  whole  Army  -  Active  Component, 
Army  Reserve,  and  Army  National  Guard. 

■  Assessment  will  start  with  Specialist/Corporal  (E4)  at  Skill  Level  1  (SL1)  and  will 
eventually  include  Sergeant  First  Class  (E7)  at  SL4. 

■  Assessment  will  be  computer-administered  via  the  Internet. 

■  Initially,  the  assessment  will  be  on  a  shared,  common  core  of  subjects  pertinent  to  all 
Soldiers  (Army-wide).  Later,  a  Military  Occupational  Specialty  (MOS)-specific 
assessment  can  be  added. 

■  Content  of  the  initial  assessment  will  include: 

o  Leadership 
o  Training 

o  Army  and  NCO  History  and  Army  Values 
o  Basic  Soldier  Skills  (Common  Tasks) 

Ultimately,  we  incorporated  all  of  the  SMA’s  guidance  into  the  DCAP,  although  the  issue 
with  the  most  immediate  impact  was  the  content  guidance.  We  determined  that  the  operational 
goal  would  be  a  test  that  could  be  administered  within  about  a  3-hour  time  frame  via  a  web- 
connected  computer.  This  translates  to  an  instrument  with  approximately  150  items.  The  test 
would  be  initially  designed  to  be  administered  to  Soldiers  in  pay  grade  E4  being  considered  for 
promotion  to  pay  grade  E5  (Sergeant).  A  single  test,  suitable  for  all  Soldiers,  regardless  of  MOS 
and  regardless  of  component  (Active  Army,  United  States  Army  Reserve,  Army  National  Guard) 
would  be  developed.  The  test  would  be  knowledge  or  knowledge-based,  essentially  in  a 
multiple-choice  format. 

The  Army  Test  Program  Advisory  Team 

Early  in  the  project  we  constituted  a  group  to  advise  on  the  operational  implications  of 
Army  assessment  testing,  primarily  as  part  of  the  needs  analysis  aspect  of  the  project. 
Simultaneously,  this  group  took  on  a  role  as  Test  Council  for  the  DCAP.  This  group  is  called  the 
Army  Test  Program  Advisory  Team  (ATP AT)  and  it  has  the  following  characteristics: 

■  It  is  made  up  of  NCOs,  mostly  in  the  Master  Sergeant  (E8)  and  Sergeant  Major  (E9) 
levels. 

■  It  includes  representatives  from  TRADOC,  HQ,  Forces  Command  (FORSCOM), 
Combined  Arms  Center  (CAC),  Center  for  Army  Leadership  (CAL),  Army  Training 
Support  Center  (ATSC),  Army  G-l,  Sergeant  Major  Academy  (USASMA),  and 
specific  organizational  representation  including  the  U.S.  Army  Armor  Center  and 
School,  the  U.S.  Army  Ordnance  Center  and  School,  and  the  13  Corps  Support 
Command  (COSCOM). 
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■  It  includes  representatives  from  the  Reserve  force  including  HQ,  Army  National 
Guard  Bureau  (ARNGB),  HQ,  Army  Reserve  Command  (USARC),  and  unit 
representatives  from  the  95th  Division  (Institutional  Training),  653rd  Area  Support 
Group  (ASG),  and  the  78th  Division  (Training  Support) 

■  It  is  co-chaired  by  two  Sergeants  Major  endorsed  by  the  ATP  AT  body.  Chairs  are 
responsible  for  determining  the  scope  and  direction  of  the  ATP AT  involvement  and 
for  setting  the  ATP  AT  meeting  agendas.  They  also  serve  as  the  point  of  contact  for 
policy  determinations  and  clarifications  when  military  representation  is  needed 
outside  of  scheduled  ATP  AT  meetings. 

■  It  has  a  flexible  membership.  Although  there  is  a  solid  core  ATPAT  group,  there  have 
been  25  individual  representatives  to  the  ATPAT. 

The  ATPAT  serves  two  distinct  purposes.  First,  it  provides  input  for  the  needs  analysis 
requirements  of  the  project,  primarily  by  providing  insight  into  operational  implications  and  real- 
world  feasibility  of  the  program.  Second,  it  serves  as  the  oversight  group  for  development  of  the 
DCAP  as  well  as  a  resource  in  identifying  and  developing  content  for  the  test.  Additionally,  the 
ATPAT  is  a  working  group  that  provides  product  reviews,  subject  matter  expertise,  and,  as 
needed,  assistance  in  the  process  of  developing  prototype  instruments  and  trial  procedures.  An 
additional  benefit  of  the  ATPAT  is  to  serve  as  a  conduit  to  explain  and  promote  the  PerformM21 
project  to  various  Army  agencies  and  constituencies. 

The  ATPAT  met  three  times  in  2003,  providing  guidance  in  four  areas: 


•  Utilization  Strategies  -  Defining  the  scope  of  the  program  and  how  the  test  will  be 
used.  That  is,  whether  and  how  it  will  be  used  in  personnel  management,  promotion, 
career  development,  training,  readiness,  retention,  and  transition. 

•  Implementation  Strategies  -  Identifying  the  steps  to  implementing,  maintaining,  and 
growing  an  Army  test,  short-  and  long-term  goals,  and  organizational  implications  to 
be  considered  in  phased  implementation. 

•  Operational  Strategies  -  Identifying  the  considerations  that  must  be  taken  into 
account  to  operationalize  an  Army-wide  testing  program  (for  developers, 
administrators,  and  users). 

•  External  Considerations  -  How  the  Army-wide  test  will  fit  in  with  other  programs 
such  as  self-development,  unit  training,  NCO  Education  System  (NCOES), 
deployments,  NCO  Evaluation  Record  (NCOER)  system,  TDA  staffing,  transition, 
Soldier  tracking  and  assignment.  Future  Force,  and  training  publications  and  updates. 

The  ATPAT  has  been  extremely  helpful  in  discussions  in  all  of  these  areas.  At  each 
meeting,  significant  portions  of  the  discussion  were  centered  on  the  nature  of  the  assessment 
(i.e.,  self-assessment  vs.  promotion).  Other  significant  discussion  and  exercise  centered  on  the 
determination  and  specification  of  the  content  domains  of  the  DCAP.  (These  are  discussed  in 
more  detail  in  the  following  sections.)  They  have  also  helped  to  identify  resources  useful  for  test 
item  development  and  field  testing. 
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Overview  of  the  DCAP  Development  Process 

In  its  general  approach,  the  DCAP  development  followed  standard  instrument 
development  steps,  as  shown  in  Figure  1.  Due  to  time  and  resource  constraints,  we  were  not  able 
to  conduct  a  typical  job  analysis  for  the  prototype  DCAP.  We 
relied  instead  on  our  collective  experience  with  the  Army  and 
guidance  from  the  ATPAT.  This  input  was  used  to  develop  a 
test  blueprint,  that  is,  a  map  of  the  test  instrument  content. 

Following  blueprint  development,  HumRRO  staff  wrote  new 
items  and  adapted  items  from  previous  research  projects  (i.e., 

Select21  and  Project  A)  (J.  Campbell  &  Knapp,  2001; 

Knapp,  2003).  SMEs  from  the  Army  and  HumRRO  reviewed 
these  items.  The  items  will  be  pilot  tested  in  2004. 

Army -Wide  Requirements  Analysis 

It  is  an  axiom  of  test  development  that  the  most  crucial 
part  of  the  process  is  the  analysis  phase,  which  identifies  what 
parts  of  the  job,  or  aspects  of  performance  should  constitute 
the  test  domain  and  subject  matter.  Preferably,  this  is 
accomplished  through  systematic  surveys  and/or  field  visits 
with  incumbents  and  supervisors  to  provide  not  only  content 
identification  but  also  measures  of  criticality  and  rankings. 

Such  detailed  and  systemic  approach  to  defining  requirements 
is  necessary  to  ensure  that  the  test  is  truly  reflective  of  Figure  1.  Steps  in  DCAP  development. 

incumbents’  job  performance.  It  directly  affects  the  utility  and 

acceptability  of  the  test.  We  did  not  do  that  for  this  initial  DCAP  -  time  and  other  resource 
constraints  placed  extreme  limitations  on  this  very  important  phase.  We  relied  on  the  SMA 
guidance  for  content  and  on  the  ATPAT  as  a  resource  to  provide  operational  interpretation  of  that 
guidance  as  well  as  application  of  job  relevant  criteria.  This  was  satisfactory  for  an  initial 
approach,  given  the  project  limitations.  It  should  not  be  seen  as  an  acceptable  long-term  approach. 

Figure  2  depicts  the  plan  for  determining  DCAP  content  requirements.  This  depicts  both  a 
long-term  concept  that  would  rely  on  survey  data  and  the  establishment  of  a  senior  NCO  Test 
Council  and  a  near-term  plan  for  the  development  of  the  prototype  DCAP  utilizing  the  ATPAT  and 
the  current  resources.  As  the  plan  indicates,  success  of  the  process  rests  heavily  with  the  ATPAT, 
which  has  provided  solid  guidance  on  content,  identified  resources  we  could  use  in  developing  the 
test,  and  provided  important  information  about  resource  limitations  and  other  constraints  we  are 
likely  to  encounter  during  the  process.  However,  it  is  also  important  to  recognize  that  the  future 
program  must  embody  a  fuller  analysis  program  as  previously  outlined.  The  arrows  in  Figure  2 
indicate  the  information  flow  generated  to  and  from  an  analytic  survey  function  that  is  essential  to 
a  long  term  program. 

During  the  analysis,  the  first  step  was  to  try  to  identify  the  tasks  or  knowledge  areas  that 
make  up  the  content  areas.  The  Army  is  task  oriented;  all  training  is  task  based  and  “task”  has  a 
very  specific,  performance  oriented  definition  and  use.  But  this  classic  and  somewhat  rigid 
definition  did  not  fit  all  of  the  content  areas,  which  differ  quite  a  bit  from  each  other: 
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Basic  Soldier  Skills  (Common  Tasks).  This  area  is  made  up  of  traditional  Army  tasks  with 
action  verb  task  statements,  conditions,  standards,  and  performance  measures.  All  tasks 
are  discrete,  observable  actions.  Moreover,  the  Army  identifies  tasks  by  the  Skill  Level 
that  is  responsible  for  acquiring  and  mastering  them  (i.e.,  SL1,  SL2).  There  are  two 
complete  doctrinal  sources  for  this  domain:  the  Soldier’s  Manual  of  Common  Tasks  Skill 
Level  1  (Department  of  the  Army,  2003),  and  the  Soldier’s  Manual  of  Common  Tasks 
Skill  Level  2,  3,  and  4  (Department  of  the  Army,  2003).  This  area  is  well  defined  and 
presented  no  problems  in  analysis  other  than  obtaining  criticality  and  importance  ratings. 


Army-Wide  Requirements  Analysis 


DCAP 


Test 

Proponent 

Director* 


Test  Council  of 
Sergeants  Major  ** 


*  Envisioned  future  organization 
**  Function  to  be  performed  initially  by 
ATP  AT  -  Later  replaced  with  duly 
constituted  council  with  established 
charter  and  authority. 
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Recommend  Test  Approaches 
Identify  Resources 


•  Select  final  test  conteni 

•  Guidance  for  future 


Organizational  Resources  1 

•  ATSC 

-CAL 

•CAC 
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•  USASMA 

•  Other  TRADOC 

-CALL 

•  Other 

EE 


SURVEY 
Incumbent  and  Supervisors 


What  do  you  do/know? 
What  should  you  do/know? 
What’s  important  to  test? 


Figure  2.  Elements  of  Army-wide  requirements  analysis. 


Leadership.  Army  leadership  is  a  broad,  somewhat  amorphous  area,  most  often  described 
in  terms  of  attributes,  exemplars,  characteristics,  anecdotes,  and  principles.  Doctrinally, 
the  primary  sources  of  leadership  information  are  the  Soldier’s  Guide  (Department  of  the 
Army,  2004),  the  Army  Noncommissioned  Officer  Guide,  (Department  of  the  Army, 
2002),  and  Army  Leadership:  Be,  Know,  Do  (Department  of  the  Army,  1999).  Most  of 
these  resources  are  broad-based  discussions  designed  to  motivate  and  inspire  as  much  as 
to  impart  knowledge.  Moreover,  leadership  is  not  Skill  Level  specific.  There  are  aspects 
of  leadership  that  are  cumulatively  acquired  over  time  and  grade  (experience)  and  are 
especially  challenging  to  impart  or  imply  boundaries,  limitations,  or  reasonable 
expectations  for  persons  at  SL1.  This  area  posed  (and  continues  to  pose)  extreme 
challenges  in  defining  and  specifying  job  relevant  knowledge  or  performance  areas. 
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Training.  Army  training  is  doctrinally  supported  by  the  documentation  in  the  Soldier’s 
Guide  (Department  of  the  Army,  2004),  the  Army  Noncommissioned  Officer  Guide , 
(Department  of  the  Army,  2002),  and  Battle  Focused  Training  (Department  of  the  Army, 

2003) .  As  an  analytic  challenge,  it  falls  somewhere  between  Basic  Soldier  Skills  and 
Leadership.  That  is,  there  are  some  specific  tasks  but  also  a  lot  of  soft  skill,  principle- 
based,  motivational  based  requirements.  Like  Leadership,  the  source  documentation  does 
not  make  any  Skill  Level  designation  or  distinction.  And  since  training,  even  at  its  lowest 
level,  is  doctrinally  an  NCO  (SL2)  requirement,  even  the  more  task-based  requirements 
have  dubious  SL1  application. 

History  and  Army  Values.  From  an  analysis  standpoint,  this  area  was  the  most  difficult. 
Doctrinally,  the  sources  for  this  area  are  the  Soldier’s  Guide  (Department  of  the  Army, 

2004)  and  the  Army  Noncommissioned  Officer  Guide  (Department  of  the  Army,  2002). 
This  area  has  all  the  problems  associated  with  Leadership,  with  which  it  shares  a  lot  of 
overlap  in  content.  Moreover,  there  is  very  little  job  relevance  to  this  area.  Army  Values, 
in  particular,  while  they  are  important  to  instill  and  to  reinforce,  are  often  exemplified  in 
performance  by  their  absence.  Army  history  also  can  easily  lead  to  identifying  and  testing 
the  trivial.  This  is  a  sensitive  area  because  of  the  commitment  of  Army  leadership  to 
reinforce  Army  values.  Identification  of  inherent  analytic  problems  is  not  meant  to 
denigrate  the  vitality  of  these  areas  in  the  Army  Soldierization  program.  However,  future 
job  analysis  should  pay  particular  attention  to  defining  this  area  based  on  job  data,  if 
possible. 

The  end  approach  to  the  areas  of  defining  Leadership  and  Training  was  for 
knowledgeable  project  staff  to  cull  the  doctrinal  literature  for  areas  that  seemed  to  provide  some 
operational  and  testable  content,  and  then  to  translate  these  into  statements  of  knowledge  areas. 
For  History/Values,  the  approach  was  somewhat  similar.  History  was  divided  into  broad  topical 
times  or  historical  events.  The  Values  are  simply  listed  as  the  seven  Army  values. 

This  approach  worked  for  the  initial  review  but  presented  problems  throughout 
development.  To  start  with,  the  listings  are  titles  only  with  no  descriptions  of  what  is  entailed 
particularly  from  a  knowledge  or  performance  area.  Moreover,  because  the  statements  are 
nonstandardized  and  not  necessarily  part  of  the  Army  performance  specifications,  reviewers 
often  put  their  own  interpretation  on  what  the  statement  involved,  particularly  when  they  thought 
about  the  statements  in  application  to  SL1  Soldiers.  Finally,  some  areas  (such  as  History)  are  just 
too  overwhelming  to  be  approached  in  this  manner;  the  idea  that,  for  example.  World  War  II  can 
be  “covered”  in  a  single  statement  borders  on  the  ludicrous.  Nonetheless,  this  approach  provided 
a  start  and,  in  the  end,  we  were  able  to  extract  testable  content  from  the  list.  But  analysis  and 
definition  of  these  domains  must  continue  and  must  be  improved,  as  suggested  by  the  long-term 
plan  depicted  in  Figure  2. 
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Blueprint  Development 


Overview 

A  test  blueprint  specifies  the  content  of  a  test  and  the  degree  to  which  each  content  area  is 
covered  (e.g.,  the  percentage  of  the  DCAP  items  that  will  assess  knowledge  of  leadership 
principles).  Blueprints  typically  reflect  the  results  of  job  analyses  and/or  the  response  of 
extremely  knowledgeable  SMEs.  In  this  case,  we  relied  on  the  input  of  the  ATP AT  to  develop  a 
solid  blueprint  for  the  DCAP.  Blueprints  are  developed  in  detail;  they  specify  the  test  approach 
by  content  area  and  also  by  subjects  or  tasks  within  each  content  area.  The  result  was  a 
comprehensive  blueprint  for  a  150-item  assessment. 

Procedure 

Prior  to  the  ATP  AT’ s  first  meeting,  HumRRO  staff  developed  draft  outlines  of  the 
material  that  might  be  covered  in  the  content  areas  of  Leadership,  Training,  and  History  and 
Values.  (These  outlines  are  in  Appendix  A.)  For  the  Common  Tasks;  the  table  of  contents  from 
the  Soldier’s  Manual  of  Common  Tasks  (SMCT)  was  used  to  provide  an  outline  of  that  content 
area.  The  ATP  AT  reviewed  these  materials  and  briefly  discussed  each  area.  Following  this 
discussion,  we  asked  the  ATPAT  members  to  weight  the  content  areas  to  determine  what 
proportion  of  the  DCAP  each  would  have.  Specifically,  we  asked  them  to  apportion  100  points 
between  the  four  areas. 

The  participants  went  through  this  activity  once,  and  then  discussed  their  rationale  for  the 
weights  they  provided  and  the  overall  results.  During  this  discussion,  the  ATPAT  indicated  that 
for  Common  Tasks,  the  test  should  include  both  Skill  Level  1  and  Skill  Level  2  items.  The 
weighting  exercise  was  then  repeated  with  the  Common  Tasks  broken  out  by  Skill  Level.  A  final 
discussion  and  weighting  was  conducted  and  we  calculated  the  average  weight  for  each  content 
area.  These  final  weights  and  the  number  of  items  are  presented  in  Table  1. 


Table  L  Test  Blueprint  Weights  Provided^  by  ATPAT 


Content  Area 

Percent  of  Test 

No.  of  items  on 
a  150-item  test 

Common  Tasks:  SL1 

46.2 

69 

Common  Tasks:  SL2 

13.5 

20 

History/ Army  Values 

15.4 

23 

Leadership 

13.1 

20 

Training 

11.9 

18 

Because  most  of  the  content  areas  were  very  broad,  there  were  more  subjects/tasks  in 
each  content  areas  than  could  be  covered  on  a  150-item  test.  It  was  therefore  necessary  to  make 
judgments  about  which  subjects/tasks  were  important  enough  to  keep  and  which  could  be 
dropped.  At  their  second  meeting,  the  ATPAT  reviewed  the  subject  areas  (see  Appendix  A)  and 
assigned  subjective  characterizations  of  Low  Value  Content  (LVC)  or  High  Value  Content 
(HVC)  to  the  sub-areas.  The  goal  was  to  eliminate  LVC  subject  areas  from  the  DCAP.  The 
criteria  to  eliminate  LVC  were  (a)  inappropriateness  for  SL1  promotion  candidates,  (b) 
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testability  of  content  in  DCAP  format,  (c)  size  (percentage)  of  the  content  area  weighting,  and 
(d)  comparison  with  HVC  subject  areas. 

In  this  discussion,  the  ATP  AT  deleted  nine  subject  areas  from  Training  and  17  subject 
areas  from  Leadership.  For  the  Common  Tasks,  they  again  reviewed  a  presentation  of  the  table 
of  contents  from  the  Soldier’s  Manual  of  Common  Tasks  (SMCT)  (SL1  and  SL2).  They  did  not 
suggest  dropping  any  items  from  the  Common  Tasks,  however  they  did  discuss  the  option  of 
tracking  for  some  Common  Tasks,  particularly  those  that  were  weapons  related  or  equipment 
specific.  In  a  tracking  approach,  it  would  be  necessary  to  identify  (and  verify)  what  item  of 
equipment  the  tested  Soldier  was  assigned  or  most  familiar  with  and  test  only  that  item. 

To  provide  the  information  for  decisions  about  test  content,  ATP  AT  members  were 
tasked  to  provide  criticality  ratings  for  each  subject  area  or  task  for  the  content  areas  of 
Leadership,  Training  and  Common  Tasks.  (History/Army  Values  was  handled  somewhat 
differently  as  discussed  below.)  The  instructions  for  this  exercise  directed  them  to  “rate  each  of 
the  subject  areas  in  terms  of  its  importance  for  inclusion  in  testing  relative  to  the  other... subject 
areas.”  They  made  these  judgments  using  a  5-point  scale  in  which  1  =  “much  less  important  than 
other  areas,”  3  =  “about  the  same  as  other  areas  ”  and  5  =  “much  more  important  than  other 
areas.”  (See  Appendix  B  for  average  ratings  for  all  subject  areas  for  the  content  areas 
Leadership,  Training,  and  Common  Tasks.) 

The  Common  Tasks  areas  presented  some  unique  challenges,  primarily  because  of  the 
scope  of  the  task  domain  (1 12  individual  tasks).  To  make  this  more  manageable,  these  individual 
tasks  were  organized  under  22  broad  “subject  areas.”  In  a  follow-up  exercise,  the  ATP  AT 
provided  additional  judgments  for  these  Common  Task  areas  and  tasks.  First,  they  weighted  each 
of  the  subject  areas  by  allocating  100  points  among  the  areas  to  determine  what  proportion  of  the 
test  should  be  devoted  to  each  area.  Twelve  SMEs  completed  the  weighting  exercise;  the  results 
of  which  are  shown  in  Appendix  C.  Second,  they  rank  ordered  the  individual  tasks  within  each 
subject  area  such  that  they  ranked  the  most  important  task  “1,”  the  second  most  important  task 
“2,”  and  so  forth.  The  results  of  these  exercises  are  shown  in  Appendix  D. 

We  had  planned  to  select  which  subject  areas  and/or  tasks  to  include  in  a  test  by  selecting 
those  that  received  a  criticality  rating  at  or  above  3.5  from  the  5-point  criticality  scale  rating 
exercise.  However,  that  cutoff  left  us  with  too  many  areas  to  cover  in  the  DCAP.  Recall  that  the 
blueprint  for  the  content  areas  (see  Table  1)  specified  the  number  of  test  items  to  be  written  for 
each  area.  For  example,  it  calls  for  18  items  on  Training.  However,  there  are  13  subject  areas 
within  Training,  which  would  mean  that  there  would  be  fewer  than  two  items  for  each  subject 
area.  Therefore,  project  staff  reviewed  the  final  items  and  further  reduced  the  number  of  subject 
areas  to  be  covered  on  the  test.  The  result  of  the  review  was  that  five  top-rated  subject  areas  were 
identified  for  Leadership  and  four  for  both  Training,  and  History/V alues.  The  ATPAT  reviewed 
this  revised  blueprint  at  their  third  meeting.  They  accepted  the  document,  with  only  one  change. 
They  replaced  “Identify  the  principles  of  BE,  KNOW,  DO”  with  “Know  the  steps  in  troop 
leading  procedures  (TLP)”  with  the  reasoning  that  the  first  is  subsumed  by  the  latter.  The  final 
blueprint  is  shown  in  Table  2. 
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Table  2.  DCAP  Blueprint 


Percentage  Number  of 
of  Test  Items 

Common  Tasks  Skill  Level  1 

46 

69 

First  Aid 

12 

M16  Rifle/M4  Carbine/M9  Pistol 

11 

Nuclear,  Biological,  &  Chemical  (NBC) 

6 

Communicate:  Radio  &  Telephone 

6 

Combat  Techniques  (Survive) 

5 

Navigate,  Mounted  &  Dismounted 

5 

Defense  Measures:  Camouflage,  SALUTE,  OPSEC 

4 

M60  Machine  Gun/M249  SAW/M240B 

4 

Individual  Conduct  &  Laws  of  War 

3 

Hand  Grenades  &  Land  Mines 

3 

Remains  Reporting  &  Handling 

2 

Caliber  .50  M2  Machine  Gun 

2 

M203  40mm  Grenade  Launcher 

2 

MK19  40mm  Grenade  Launcher  Machine  Gun 

2 

Ml 36  Launcher  AT4  Light  Anti-Tank  Weapon 

2 

Common  Tasks  Skill  Level  2 

14 

20 

Combat  Techniques  (Survive) 

4 

First  Aid:  MEDEVAC,  Preventive  Medicine 

4 

Equipment  Checks:  PMCS,  Supply  Discipline,  Property  Accountability 

3 

Defense  Measures:  Squad  Defense 

3 

Navigate:  Map  Overlays 

2 

A 

Risk  Management:  Accident  Prevention 

2 

Nuclear,  Biological,  &  Chemical  (NBC) 

2 

History/Values 

15 

23 

Army  values 

7 

Courtesy  &  customs 

/ 

c 

Volunteer  Army 

J 

A 

End  of  Cold  War 

4 

Leadership 

13 

20 

Identify  the  leadership  duties,  responsibilities,  authority,  &  requirements  of 

4 

officers  &  NCOs 

Know  the  policies  &  procedures  of  the  chain  of  command  &  of  the  NCO 

4 

support  channel 

Know  the  steps  in  Troop  Leading  Procedures  (TLP) 

4 

Know  the  principles  of  discipline 

4 

Identify  the  risk  management  process 

4 

Training 

12 

18 

Train  subordinates 

5 

Prepare  for  and  conduct  preparatory  marksmanship  training  (PMT) 

5 

Identify  the  roles  &  responsibilities  of  the  NCO  in  training 

4 

A 

Prepare  for  and  conduct  drill  &  ceremonies 

4 
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Item  Tracking 

One  important  factor  considered  in  item  development  and  administration  is  tracking. 
Tracking  is  an  assessment  technique  wherein  a  set  of  items  (i.e.,  a  module)  is  administered  to  a 
subset  of  the  participants.  The  ATP  AT  suggested  considering  tracking  for  some  weapons  tasks 
within  the  Common  Tasks  because  not  all  Soldiers  will  be  equally  familiar  with  all  weapons,  and 
it  would  be  unfair  to  assess  their  knowledge  of  a  weapon  with  which  they  have  little  or  no 
experience.  It  might  be  necessary  to  track  other  areas,  depending  on  a  Soldier’s  assignment  or 
unit.  One  method  to  handle  tracking  would  be  to  ask  participants  to  complete  a  computerized 
background  information  form  prior  to  beginning  the  assessment.  This  form  could  ask  Soldiers, 
among  other  things,  whether  they  use  various  personal  and/or  crew-served  weapons.  For 
example,  armor  Soldiers  (MOS  19K/19D)  typically  carry  an  M9  pistol  rather  than  an  M16  rifle 
as  a  personal  weapon.  If  they  indicate  this  on  the  background  form,  they  will  be  presented  a  test 
module  about  the  use  and  maintenance  of  the  M9  rather  than  the  Ml 6. 

However,  the  decision  to  permit  tracking  is  not  a  trivial  one.  Tracking  is  primarily  useful 
under  conditions  where  the  (a)  test  is  being  used  as  a  criterion,  and  (b)  tracking  can  be  done  by 
unit.  If  individuals  were  allowed  to  decide  which  track  to  take  (particularly  in  an  operational 
assessment),  the  situation  could  present  some  unknown  implications  about  test  equivalency  and 
comparability  and  whether  there  are  advantages  to  taking  one  track  or  another.  For  example, 
there  are  not  necessarily  comparable  or  equivalent  choices  to  be  made  between  many  weapons 
(i.e.,  it  may  not  be  an  either/or  situation).  This  is  complicated  by  the  fact  that  not  all  Soldiers  will 
have  a  similar  level  of  experience  with  the  weapons  they  nominally  use. 

The  goal  is  to  avoid  tracking  in  the  core  test.  One  approach  is  to  develop  items  on  such 
basic  knowledges  that  all  Soldiers  could  be  expected  to  have  a  certain  level  of  familiarity 
regardless  of  the  particular  conditions  of  their  assignment.  We  will  also  seek  to  collect  so-called 
“recency  and  frequency”  information  during  the  pilot  to  determine  the  levels  of  examinee 
involvement  with  all  tasks.  This  will  provide  needed  information  for  determining  tracking 
guidance  for  an  operational  test. 

Leadership  Judgment  Exercise  (LeadEx) 

Although  it  has  not  been  included  in  the  test  blueprint,  we  also  intend  to  administer  a 
situational  judgment  test  that  was  developed  for  the  NC021  project  (Ford  et  al.,  2000;  Knapp  et 
al.,  2002)  -  the  Leadership  Judgment  Exercise  (LeadEx).  The  LeadEx  was  shown  to  be  predictive 
of  success  at  the  E5  and  E6  pay  grades;  that  is  Soldiers  who  performed  well  on  the  LeadEx  also 
were  rated  highly  on  performance  assessment  evaluations  obtained  from  their  supervisors.  A 
sample  item  is  shown  in  Figure  3.  The  LeadEx  assesses  eight  performance  dimensions: 

1.  Problem  Solving  and  Decision  Making  Skill 

2.  Motivating,  Leading,  and  Supporting  Subordinates 

3.  Directing,  Monitoring,  and  Supervising  Work 

4.  Training  Others 

5.  Relating  to  and  Supporting  Peers 

6.  Team  Leadership 

7.  Concern  for  Soldier  Quality  of  Life 

8.  Cultural  Tolerance 
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Instructions:  For  each  item,  mark  which  course  of  action  you  would  be  MOST 
likely  to  follow  with  an  “ M "  and  mark  the  choice  you  would  be  LEAST  likely  to 
choose  with  an  “L. " 

As  a  junior  NCO,  you  need  to  counsel  a  subordinate.  What  would  be  your  priority 
when  preparing  for  and  conducting  the  counseling? 

_ a.  Prepare  a  course  of  action  that  you  want  the  Soldier  to  follow 

_ b.  Plan  to  guide  and  encourage  the  Soldier  to  arrive  at  his  own  solutions 

_ c.  Focus  on  the  sanctions  and  rewards  that  you  control 

_ d.  Follow  the  outline  of  the  DA  for  4856-R,  General  Counseling  Form 

Figure  3.  Sample  LeadEx  item. 


Item  Development  and  Review 

Procedure 

HumRRO  staff  used  a  variety  of  resources  to  write  the  DCAP  items,  including  field 
manuals,  training  materials,  and  SMEs.  A  list  of  doctrinal  publications  used  in  development  is  in 
Table  3.  Project  staff  developed  items  based  on  these  resources  and  also  incorporated  items  from 
related  projects  (e.g.,  Project  A  and  Select21).  All  DCAP  items  were  subjected  to  an  iterative 
review  process,  including  both  internal  and  external  reviews.  These  reviews  considered  the 
currency  of  item  content  (e.g.,  in  terms  of  technology  and/or  procedures)  and  how  well  each  item 
adhered  to  the  task  requirement.  HumRRO  project  staff  implemented  revisions,  updated  items, 
and  wrote  additional  questions  as  necessary  to  replace  dropped  items. 


Table  3.  Resources  Used  for  hem  Development 


-*•  **'*-’■ -  ~  J  ~  i r 

Title 

Manual/Publication 

Publication  Date 

The  Army  Noncommissioned  Officer  Guide 

FM  7-22.7 

23  December  2002 

Army  Leadership  Be,  Know,  Do 

FM  22-100 

August  1999 

Battle  Focused  Training 

FM  7-1 

15  September  2003 

Drill  and  Ceremonies 

FM  3-21.5 

7  July  2003 

Flags,  guidons,  streamers,  tabards,  and 
automobile  and  aircraft  plates 

Army  Regulation  840-10 

1  November  1998 

BET  Soldier’s  Handbook 

TRADOC  Pamphlet  600-4 

1  April  2001 

Rifle  Marksmanship  M16A1,  M16A2/3, 
M16A4,  and  M4  Carbine 

FM  3-22.9 

24  April  2003 

Risk  Management 

FM  100-14 

23  April  1998 

Salutes,  Honors,  and  Visits  of  Courtesy 

Army  Regulation  600-25 

1  September  1983  • 

The  Soldier’s  Guide 

FM  7-21.13 

February  2004 

Soldier’s  Manual  of  Common  Tasks  Skill 
Level  1 

Soldier  Training  Publication 
N0.21-1-SMCT 

February  2003 

Soldier’s  Manual  of  Common  Tasks  Skill 
Levels  2,  3,  and  4 

Soldier  Training  Publication 
No.  21-24-SMCT 

February  2003 

Training  the  Force 

FM  7-0 

22  October  2002 
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Content  for  the  History/ Army  Values  section  of  the  assessment  was  based  primarily  on 
the  History  and  Courtesies  and  Customs  chapters  of  the  draft  Soldier’s  Guide  as  these  were  the 
most  relevant  to  this  section.  The  content  for  the  History  section  was  not  developed  in  the  same 
manner  as  the  other  sections.  Instead,  an  outline  (see  Appendix  A)  was  presented  to  the  ATP  AT 
to  get  their  input  as  to  what  topics  should  be  emphasized  consistent  with  their  earlier  weighting 
of  this  content  area  (15.4%  of  the  DCAP).  The  ATP  AT  suggested  we  stay  at  a  relatively  broad 
level.  In  particular  they  wanted  Courtesies  and  Customs  and  Army  Values  to  account  for  half  of 
the  History/ Army  Value  test  content  area.  Specific  recommendations  included  items  concerning 
the  purpose  of  the  hand  salute  and  proper  display  of  the  flag.  For  Army  Values,  vignettes  or 
definitional  items  were  suggested.  The  ATP  AT  wanted  the  other  half  of  the  assessment  to  cover 
the  Army  Birthday,  the  volunteer  Army,  the  end  of  the  Cold  War,  and  the  attacks  on  1 1 
September  2001.  Supplemental  resources  for  the  History  items  include  the  IET  Soldier’s 
Handbook;  Salutes,  Honors,  and  Visits  of  Courtesy;  and  Drill  and  Ceremonies. 

SMEs  from  the  NCO  Academies  at  Fort  Knox  and  Fort  Eustis  reviewed  the  Common 
Tasks,  Flistory/Army  Values,  Leadership,  and  Training  sections’  items.  We  also  targeted  the 
Center  for  Army  Leadership  (CAL),  the  Military  History  Instructional  Support  Team  (MHIST), 
and  the  United  States  Army  Sergeants  Major  Academy  (USASMA)  as  the  primary  reviewers  in 
the  areas  of  History/ Army  Values,  Leadership,  and  Training  content  areas.  Table  4  summarizes 
the  DCAP  review  process. 


Table  4.  Summary  of  PerformM2 1  Item  Review^ 


Content  Area 

Internal 

HumRRO 

Ft.  Eustis 
NCOA 
SMEs 

Ft.  Knox 
NCOA 
SMEs 

CAL/CSI/ 

MHIST 

SMEs 

USASMA/ 

TRADOC 

Staff 

Seiect21  SMEs 

SL1 

X 

X 

X 

X* 

SL2 

X 

X 

X 

History/Values 

X 

X 

X 

X 

X 

Training 

X 

X 

X 

X 

X 

Leadership 

X 

X 

X 

X 

X 

Note.  CAL  =  Center  for  Army  Leadership,  CSI  =  Combat  Study  Institute,  NCOA  =  Noncommissioned 
Officer’s  Academy,  MHIST  =  Military  History  Instructional  Support  Team,  USASMA  =  United  States 
Army  Sergeants  Major  Academy. 

*  Items  included  in  the  DCAP  separately  reviewed  by  SME  during  the  past  10  months  as  part  of  the 
Select21  Project. 


Currently,  the  DCAP  item  bank  contains  274  items.  The  DCAP  is  intended  to  be  a  150-item 
assessment.  It  is  anticipated  that  based  on  the  results  of  the  pilot  tests,  some  items  will  be  dropped. 
Also,  to  take  advantage  of  computer  based  testing,  it  is  possible  to  deviate  from  the  traditional  four- 
choice,  one-correct  format  of  traditional  paper-based  testing  and  incorporate  matching,  “drag  and 
drop,”  “select  all  that  apply,”  and  ranking  items.  The  result  is  that  some  items  are  worth  more  points 
than  others.  An  example  of  this  computer-based  approach  is  shown  in  Figure  4. 

For  this  sample  item,  a  possible  scoring  scheme  would  have  a  Soldier  earn  one  point  for 
each  correct  option  selected  and  one  point  for  each  incorrect  option  not  selected  up  to  a 
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maximum  of  nine  points.  Partial  credit  could  be  available  as  well.  For  example,  a  Soldier  who 
selected  options  a,  b,  and  g  would  earn  four  points — one  each  for  correctly  selecting  options  a 
and  b,  and  one  each  for  correctly  not  selecting  options  h  and  i.  As  shown  in  Table  5,  the  existing 
274  items  have  a  potential  of  405  scoring  points.  However,  final  scoring  designs  have  not  yet 
been  decided;  they  must  also  consider  reliability  weights  and  coverage.  For  example,  having  a 
single  item  worth  a  large  number  of  points  relative  to  other  items  could  adversely  impact  the 
reliability  of  the  test.  A  number  of  different  options  will  be  explored. 


When  making  a  mission  risk  assessment,  what  factors  should  be  considered? 
Select  all  that  apply. 

a.  Weather 

b.  Inherent  dangers  of  equipment  to  be  used 

c.  Troop  training  and  proficiency 

d.  Environmental  concerns 

e.  Complexity  of  the  mission 

f  Protective  equipment  and  capabilities 

g.  Enemy  capabilities 

h.  Other  courses  of  action  (COA)  available 

i.  Gender  of  troops 

*  Italicized  options  are  correct. 

Figure  4.  Sample  non-traditional  DCAP  item. 


Table  5.  DCAP  Item  and  Point  Distribution 


Content  Area 

No.  of  Items  in 
Item  Bank 

No.  of  Points 

Common  Tasks:  SL1 

152 

192 

Common  Tasks:  SL2 

38 

57 

History/Army  Values 

22 

27 

Leadership 

33 

69 

Training 

29 

60 

Total 

274 

405 

Supporting  Software  and  Computer  Systems 

The  DCAP  will  be  computer  administered  to  Soldiers  at  proctored  sites  across  CONUS 
and  overseas.  After  comparing  many  systems,  HumRRO  selected  Questionmark’s  Perception® 
as  the  best  test  development  and  test  administration  product  on  the  market  to  meet  the  program’s 
needs.  From  a  test  development  perspective,  Perception®  provides  several  capabilities  that  make 
it  an  ideal  platform  for  computerized  knowledge  and  skill  assessment.  First,  it  has  the  capability 
to  use  a  variety  of  question  types,  including  multiple-choice,  matching,  ranking,  Likert-type 
scales,  and  identifying  appropriate  sequences  of  steps.  In  addition,  the  software  allows  use  of 
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graphics,  illustrations,  photographs,  and  video  clips.  The  use  of  pictures  and  video  reduces  the 
reading  level  required  to  take  the  exam  and,  consequently  reduces  the  risk  of  adverse  impact 
often  found  with  use  of  job  knowledge  tests.  It  also  increases  the  fidelity  of  the  items,  making  the 
questions  more  performance  oriented.  For  example,  it  is  possible  to  ask  a  respondent  to  drag  and 
drop  pieces  to  “build”  a  unit  symbol.  That  makes  it  possible  to  tap  a  Soldier’s  ability  to  apply 
knowledge  rather  than  just  to  select  the  correct  symbol  from  a  group  presented.  Perception®  is  a 
commercial  off-the-shelf  system  (COTS)  which  HumRRO  has  purchased  but  which  would  be 
available  to  the  Army  or  other  test  developers  in  the  future. 

The  other  aspect  of  a  computer  based  test  system  is  the  delivery  system  which  would 
provide  a  portal  to  the  DCAP  test  during  administration.  We  concentrated  on  existing  Army 
systems  and  identified  the  Digital  Training  Facilities  (DTF)  which  are  part  of  the  The  Army 
Distributed  Learning  Program  (TADLP)  and  administered  out  of  the  Army  Enterprise 
Management  Center  at  Fort  Eustis,  Virginia.  The  DTFs  are  located  worldwide  and,  while  locally 
administered  and  managed,  are  generally  available.  A  typical  DTF  consists  of  16  computer  work 
stations  with  Internet  access.  We  have  completed  preliminary  coordination  to  utilize  DTFs  as  a 
portal  for  the  DCAP  pilot  tests  during  2004. 

Plan  for  Pilot  Testing 


Overall  Test  Administration 

As  we  move  into  Phase  II  of  the  PerformM21  research  program  (Design  Selection  and 
Prototype  Measure  Development  and  Testing),  the  DCAP  core  assessment  will  be  pilot  tested. 
The  pilot  test  is  slated  to  begin  in  early  2004  and  will  target  a  test  sample  of  600  to  1000  Soldiers 
with  representation  from  the  Active  Army,  the  Army  National  Guard,  and  United  States  Army 
Reserve.  ARI  has  submitted  Research  Support  Requests  (RSR)  asking  for  Soldiers  in  pay  grade 
E4  with  24  or  more  months  in  service  from  a  broad  sample  of  MOS.  At  this  point,  ARI  is  in 
negotiations  for  the  required  troop  support.  Issues  associated  with  large-scale  troop  deployments 
are  complicating  the  process. 

The  plan  is  to  deliver  the  DCAP  pilot  assessment  via  the  Internet.  Soldiers  tasked  to 
participate  in  the  research  will  be  instructed  to  schedule  a  time  (within  a  specified  time  window) 
to  go  to  their  local  DTF  to  take  the  assessment.  Coordination  with  local  DTF  managers  at  each 
pilot  test  location  is  a  requirement.  Because  DTFs  are  primarily  established  to  serve  a  training 
function,  using  them  for  a  test  function  will  require  some  adjustments.  For  example,  the  pilot 
DCAP  tests  must  be  proctored,  either  with  project  staff  or  NCOs  or  other  personnel  specifically 
tasked  and  trained  for  that  function. 

In  contrast  to  most  research  Soldier  taskings,  Soldiers  participating  in  the  DCAP  pilot  test 
must  be  identified  several  weeks  before  the  pilot  test  is  administered.  This  is  needed  because  (a) 
Soldiers  need  to  register  with  the  DTF  beforehand  and  the  DTF  must  schedule  the  DCAP  into 
their  availability  requirements,  and  (b)  we  need  to  provide  the  selected  Soldiers  with  a  DCAP 
test  preparation  guide. 
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Solider  Preparation 


The  goal  of  the  pilot  is  to  replicate,  as  much  as  possible,  potentional  operational 
conditions.  Although  individual  DCAP  results  will  not  be  used  or  reported  operationally,  we 
intend  to  give  Soldiers  adequate  information  to  prepare  themselves  for  the  test  as  if  it  were  a  “for 
real”  examination  including  sample  items  and  test  references.  The  intent  is  to  provide  this 
information  on-line  using  an  Army  Knowledge  Online  (AKO)  link. 

Soldier  Feedback 

We  will  ask  pilot  test  participants  for  their  reactions  to  the  various  aspects  of  the 
assessment  program.  This  will  include  the  assessment  instrument  itself,  the  use  of  DTFs  as 
administration  centers,  the  ease  of  use  of  the  software,  and  the  usefulness  of  the  test  preparation 
materials.  We  will  incorporate  their  feedback  in  revising  the  various  aspects  of  the  program. 

Data  Analysis 

After  the  pilot  test  administration,  we  will  examine  the  item  statistics  (e.g.,  percent  correct, 
point-biserial  correlations)  for  each  item.  Items  exhibiting  poor  item  statistics  will  be  flagged  for 
review  and  modification  or  deletion.  We  will  also  analyze  the  data  to  determine  whether  there  are 
differences  in  scores  associated  with  gender,  race/ethnicity,  and  with  service  component  (Active 
Army,  Army  National  Guard,  United  States  Army  Reserve).  We  will  also  analyze  the  results  by 
job,  most  likely  by  collapsing  MOSs  into  combat,  combat  support,  and  combat  service  support 
classifications.  We  will  compute  reliability  estimates  for  the  entire  assessment,  as  well  as  for 
sections  of  the  instrument  (e.g.,  Common  Tasks,  History).  We  will  also  examine  the  correlation 
between  scores  on  the  four  test  sections,  particularly  Leadership  and  LeadEx  scores. 

Conclusions 

The  careful  development  of  a  blueprint  for  the  content  areas,  subject  areas,  and  tasks 
insures  that  the  DCAP  will  cover  the  information  that  is  important  for  Soldiers  to  know  if  they 
are  to  be  promoted.  Targeted  reviews  by  subject  matter  experts  (e.g.,  NCO  Academy  staff, 
content  specialists,  the  ATPAT)  ensures  that  the  items  are  clearly  written,  are  correctly  keyed, 
are  verifiable,  and  cover  appropriate  topics.  The  use  of  state-of-the-art  software  to  deliver  the 
assessment  allows  a  high-fidelity  presentation  of  the  items,  allows  candidates  to  demonstrate 
both  knowledge  of  an  area  and  the  ability  to  apply  that  knowledge  in  job-like  conditions. 

We  will  continue  to  develop  the  details  for  the  pilot  under  Phase  II.  The  assessment  that 
will  exist  after  the  pilot  test  still  will  not  be  the  same  as  we  would  recommend  for  a  long-term 
program.  Because  of  resource  constraints,  the  blueprint  is  based  on  an  abbreviated  process  rather 
than  a  thorough  job  analysis.  It  will  be  important  to  revisit  this  information  as  we  move  forward 
with  the  project. 
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Appendix  A 

Outlines  of  Content  Areas 


DCAP  HISTORY/ARMY  VALUES  SECTION  SUBJECT  AREAS: 

1.  Colonial  Times 

•  The  history  behind  militias  in  the  U.S. 

o  The  first  militia  regiments,  (e.g..  Where  were  they?  Do  they  still  serve?) 
o  The  militia  system  of  defense. 

•  The  Revolutionary  War 

o  Where  did  it  start? 

o  Involvement  of  other  nations  (e.g.,  France). 

o  Notable  Soldiers/commanders  (e.g.,  Washington,  Hale,  Marion,  Lafayette), 
o  Significant  battles  (e.g..  Battles  of  Saratoga,  Cowpens). 

•  The  War  of  1812 

o  What  was  over-riding  strategy  (e.g.,  push  British  out  of  Canada)? 
o  Most  significant  battle?  (e.g.,  Battle  of  New  Orleans) 

•  The  War  with  Mexico 

o  The  Alamo 

o  Focus  of  the  dispute  (Texas) 

2.  Civil  War  to  WWI 

•  The  Civil  War 

o  Slavery’s  role  (e.g.,  Lincoln’s  platform  that  he  wouldn’t  extend  it  into  territories 
led  to  the  secession  of  SC  with  his  election), 
o  Famous  Soldiers/commanders  (e.g.,  Lee,  Scott,  Grant,  Jackson), 
o  Famous  battles  (e.g.,  Antietam,  Shiloh,  Gettysburg). 

•  The  Frontier  Settlements 

o  Protecting  settlers  -  pushing  Native  Americans  onto  reservations 
o  Battle  at  the  Little  Big  Horn 

o  Exploring  Alaska  before  formation  of  civilian  government 

•  The  Spanish-American  War 

o  Intervention  in  Cuba’s  war  of  liberation  from  Spain 
o  What  happened  to  get  Congress  to  declare  war? 

3.  The  World  Wars  and  Containment 

•  WWI 

o  How  did  it  start? 

o  What  prompted  U.S.  to  get  involved? 
o  Famous  battles  (e.g.,  Ardennes  -  Rock  of  Marne), 
o  Notable  commanders  (e.g.,  GEN  John  J.  “Black  Jack”  Pershing), 
o  The  significance  of  V-Day. 
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•  Between  the  Wars 

o  National  Defense  Act  of  1920 
o  Stability  operations  in  Soviet  Russia  and  Siberia 
o  Naval  limitations  treaty  with  Japan 

o  Germany  re-arms,  Mussolini  attacks  Ethiopia,  revolution  in  Spain 

•  wwn 

o  What  led  to  WWII? 

o  What  led  to  U.S.  involvement?  (e.g.,  Pearl  Harbor) 
o  Famous  Battles  (e.g.,  D-Day,  Battle  of  the  Bulge,  Battle  of  Okinawa), 
o  Atomic  bombs 

o  Notable  leaders  (e.g.,  MacArthur,  Eisenhower,  Patton). 

•  The  Cold  War 

o  Struggle  of  power  against  Soviet  Union  and  international  communism 

•  Korea 

o  What  led  to  U.S.  involvement? 

•  Vietnam 

o  What  led  to  escalation  of  U.S.  commitment? 

4.  Post- Vietnam  and  the  Volunteer  Army 

•  The  Volunteer  Army 

o  Grenada 
o  Panama 

•  The  End  of  the  Cold  War 

•  The  Gulf  War 

o  What  led  to  this? 

o  Members  of  coalition  that  defeated  Iraq? 

•  Relief  in  Africa 

o  What  led  to  U.S.  involvement? 
o  What  was  resolution? 

•  Haiti 

o  What  led  to  U.S.  involvement? 
o  What  was  resolution? 

•  The  Balkans 

o  What  led  to  U.S.  involvement? 
o  What  was  resolution? 

•  The  War  on  Terrorism 

o  September  11,  2001 
o  Operation  Enduring  Freedom 

•  The  Second  Gulf  War  (Operation  Iraqi  Freedom) 

5.  The  Contemporary  Operational  Environment 

•  Increasing  range  of  threats 

o  Smaller,  low-tech  opponents  using  guerrilla/terrorist  methods  -  to  large, 
modernized  forces 
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•  Variables  affecting  operational  environments 

o  Physical;  nature  and  stability  of  the  state;  military  capabilities;  technology; 
information;  external  organizations;  sociological  demographics;  regional/global 
relationships;  national  will;  time;  economics 

•  Full  spectrum  operations 

o  Offense 
o  Defense 
o  Stability 
o  Support 

•  Homeland  Security 

•  The  Army'Transformation 

6.  Courtesies  and  Customs 

•  The  history  behind  the  hand  salute 

o  From  late  Roman  times  to  today 

•  The  history  behind  the  bugle  call 

o  Use  in  communication 
o  Twilight  Tattoo — Taps 

•  The  history  behind  and  colors  of  the  Army  Flag 

o  First  U.S.  Army  flag  unfurled  in  1956  in  Philadelphia 
o  Red,  white,  &  blue  selected  as  colors  of  U.S.  flag 

o  History  of  symbols  (e.g.,  implements  of  warfare;  the  pike;  the  drum/drumstick; 
Phrygian  cap;  coiled  rattlesnake) 

o  Designed  to  meet  the  need  for  one  banner  to  represent  the  entire  Army 

7.  The  Seven  Army  Values 

•  Historical  and  contemporary  examples  of  what  makes  up  each  of  the  Army  Values. 
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DCAP  LEADERSHIP  SECTION  SUBJECT  AREAS 

1 .  Identify  the  leadership  duties,  responsibilities,  authority,  and  requirements  of  officers, 
warrant  officers,  noncommissioned  officers,  and  Department  of  the  Army  civilians. 

2.  Know  the  problem  solving  steps  in  decision  making. 

3.  Know  the  steps  in  troop  leading  procedures  (TLP). 

4.  Know  and  apply  the  backwards  planning  process. 

5.  Know  the  principles  of  discipline. 

6.  Identify  team  and  group  organizations  and  doctrine. 

7.  Know  the  principles  of  team  building 

8.  Know  the  characteristics  of  effective  counseling. 

9.  Enforce  compliance  with  the  Army’s  equal  opportunity  and  sexual  harassment  policies. 

10.  Identify  the  legal  implications  of  the  Army’s  homosexual  conduct  policy. 

11.  Comply  with  and  enforce  the  Uniform  Code  of  Military  Justice  (UCMJ) 

12.  Comply  with  the  Laws  of  War  and  the  Geneva  and  Hague  Conventions. 

13.  Comply  with  the  requirements  of  the  Code  of  Conduct. 

14.  Understand  and  enforce  the  principles  of  rules  of  engagement  (ROE)  and  use  of  force. 

15.  Comply  with  host  nation,  federal,  state,  and  local  environmental  laws  and  regulations. 

16.  Support  unit  and  family  readiness  through  the  Army  Family  Team  Building  Program. 

17.  Know  the  requirements  of  the  Army  family  and  dependent  responsibility  policy. 

18.  Know  the  principles  of  financial  management. 

19.  Resolve  an  ethical  problem. 

20.  Interact  with  the  news  media. 

21.  Write  in  the  Army  style. 

22.  Conduct  a  risk  assessment  analysis. 

23.  Know  the  planning  principles  and  application  of  METT-TSL  (mission,  enemy,  terrain, 
troops  available  -  time,  space,  logistics). 
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24.  Know  the  content  and  organization  of  an  operations  order  (OPORD). 

25.  Prepare  and  issue  a  warning  order  (WARNO). 

26.  Prepare  and  issue  an  oral  OPORD. 

27.  Prepare  and  issue  a  fragmentary  order  (FRAGO). 

28.  Know  the  process  and  content  of  the  professional  development  structure  and  the 
Noncommissioned  Officer  Education  System  (NCOES). 

29.  Know  and  apply  the  policies  and  procedures  of  the  chain  of  command  and  of  the 
noncommissioned  officer  support  channel. 

30.  Know  the  Charge  to  the  NCO,  the  NCO  Vision,  and  the  NCO  Creed. 

31.  Know  the  function  and  use  of  the  Enlisted  Records  and  Evaluation  Center  (EREC). 

32.  Know  the  requirements  and  application  of  the  decentralized  (E2-E4)  and  the  semi- 
centralized  (E5-E6)  promotion  systems. 

33.  Know  the  purpose,  format,  and  functioning  of  the  Noncommissioned  Officer  Evaluation 
Record  (NCOER)  system. 

34.  Identify  the  principles  of  BE,  KNOW,  DO. 

35.  Identify  the  steps  of  assuming  a  leadership  position. 

36.  Know  Soldier  recognition  doctrine  and  principles. 

37.  Prepare  for  and  conduct  inspections  and  on-the-spot  corrections. 
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DCAP  TRAINING  SECTION  SUBJECT  AREAS 


1.  Identify  the  roles  and  functions  of  initial  training,  institutional  training,  self-development 
training,  professional  military  education,  and  unit  training. 

2.  Identify  the  roles  and  responsibilities  of  the  noncommissioned  officer  in  training. 

3.  Identify  unit  training  sequences,  cycles,  and  events. 

4.  Know  the  principles  and  applications  of  Army  training  time  management  (Green,  Amber, 
Red). 

5.  Identify  the  organization,  purpose,  and  relationships  of  the  Army  Universal  Task  List 
(AUTL),  mission-essential  task  lists  (METL),  collective  tasks,  Soldier  tasks,  and  common 
tasks. 

6.  Identify  the  content,  uses,  sources,  and  locations  of  training  publications  [Soldier  training 
publications  (STP),  Army  Training  and  Evaluation  Program  Mission  Training  Plans 
(ARTEP-MTP),  field  manuals  (FM),  technical  manuals  (TM),  training  support  packages 
(TSP),  tactics,  techniques,  and  procedures  (TTP),  lessons  learned  (LL)]. 

7.  Identify  categories,  uses,  and  sources  of  Training  Support  System  (TSS)  technologies: 
training  aids,  devices,  simulations,  and  simulators  (TADSS). 

8.  Conduct  training  planning 

9.  Conduct  training  preparation. 

10.  Prepare  for  and  conduct  Sergeant’s  Time  and  opportunity  training. 

1 1.  Prepare  for  and  conduct  training  assessment. 

12.  Know  the  10  principles  of  Battle  Focused  Training. 

13.  Prepare  a  training  outline/lesson  plan. 

14.  Conduct  performance  oriented  training. 

15.  Prepare  for  and  participate  in  training  meetings. 

16.  Prepare  for  and  present  classroom  instruction. 

17.  Prepare  for  and  conduct  battle  drills  and  crew  drills. 

18.  Prepare  for  and  conduct  drill  and  ceremonies. 
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19.  Prepare  for  and  conduct  preparatory  marksmanship  training  (PMT). 

20.  Train  subordinates  to  perform  an  individual  task. 

21.  Conduct  team  training/train  a  team. 

22.  Prepare  and  maintain  a  Leader  Book. 
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Appendix  B 

Criticality  Ratings  for  DCAP  Subject  Areas 


Subject  Area 


Mean1 

SD2 

Common  Tasks:  SL1 

4.78 

0.67 

First  Aid 

4.67 

0.50 

Communicate:  Radio  and  Telephone 

4.56 

0.73 

M16  Series  Rifle 

4.44 

0.88 

Nuclear,  Biological,  and  Chemical  (NBC) 

4.33 

0.71 

Combat  Techniques  (Survive) 

4.22 

0.67 

Defense  Measures:  Camouflage,  SALUTE,  OPSEC 

3.67 

1.12 

M4  Series  Carbine  (Ml 6  Variant) 

3.67 

0.71 

Hand  Grenades  and  Land  Mines  (M18A1  Claymore) 

3.56 

1.24 

Navigate,  Mounted  and  Dismounted 

3.56 

1.01 

M9  9mm  Beretta  Pistol 

3.44 

1.42 

M60  Medium  Machine  Gun 

3.44 

1.13 

M249  Light  Machine  Gun  Automatic  Rifle 

3.33 

1.12 

M136  Launcher  AT4  Light  Anti-Tank  Weapon 

3.22 

1.30 

Caliber  .50  M2  Machine  Gun 

3.22 

1.30 

M203  40  mm  Grenade  Launcher  (Ml 6  Series) 

3.11 

1.36 

M240B  Medium  Machine  Gun 

3.11 

1.05 

Remains  Reporting  and  Handling 

3.00 

1.50 

MK19  40mm  Grenade  Launcher  Machine  Gun 

3.00 

1.12 

Individual  Conduct  and  Laws  of  War 

2.67 

1.00 

Crowd  Control:  Riot  Baton  and  Formations 

2.44 

1.24 

See:  Surveillance  and  Identification 

Common  Tasks:  SL2 

5.00 

0.00 

First  Aid:  MEDEVAC,  Preventive  Medicine 

4.67 

0.71 

Nuclear,  Biological,  and  Chemical  (NBC) 

4.56 

0.88 

Risk  Management:  Accident  Prevention 

4.56 

0.73 

Combat  Techniques  (Survive) 

4.56 

0.73 

Equipment  Checks:  PMCS,  Supply  Discipline 

4.44 

0.73 

Navigate:  Map  Overlays 

3.89 

0.60 

Defense  Measures:  Squad  Defense 

3.56 

1.01 

Security  and  Control 

3.44 

1.01 

Unit  Operations:  FRAGO,  WARNO,  Unit  Movement 

3.00 

0.87 

Remains  Reporting  &  Handling 

2.78 

1.20 

Administration/Management:  Awards,  NCOER,  etc. 

2.78 

0.83 

Individual  Conduct  and  Laws  of  War 

1  Means  reflect  the  averaged  rating  on  a  5  point  scale  where  1  =  lowest  criticality  and  5  =  highest  criticality. 

2  Standard  Deviation.  This  reflects  how  much  raters  differed  individually  from  the  reported  mean  rating.  Low  SD 
numbers  (approaching  zero)  indicate  raters  gave  the  same  or  close  to  the  same  rating.  Higher  SD  numbers  indicate 
individual  raters  differed  more  on  their  ratings. 
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Average 

criticality  rating  and  standard  deviation  for  DCAP  subject  areas  (continued) 

Mean 

SD 

Training 

4.73 

1.49 

Train  subordinates  to  perform  an  individual  task. 

4.73 

2.24 

Prepare  for  and  conduct  preparatory  marksmanship  training  (PMT). 

4.55 

0.82 

Identify  the  roles  and  responsibilities  of  the  NCO  in  training. 

4.27 

2.15 

Prepare  for  and  conduct  drill  and  ceremonies. 

4.18 

1.66 

Conduct  performance  oriented  training. 

4.18 

1.83 

Prepare  for  and  conduct  crew  drills. 

4.09 

2.95 

Identify  the  roles  and  functions  of  initial  training,  institutional 
training,  self-development  training,  professional  military  education, 
and  unit  training. 

4.09 

3.14 

Know  the  principles  and  applications  of  Army  training  time 
management  (Green,  Amber,  Red). 

3.73 

1.27 

Identify  the  content,  uses,  sources,  and  locations  of  training 
publications,  ARTEP-MTP,  field  manuals,  technical  manuals, 
training  support  packages,  tactics,  techniques,  and  procedures, 
lessons  learned. 

3.64 

1.12 

Know  the  principles  of  Battle  Focused  Training. 

3.64 

1.36 

Conduct  training  preparation. 

3.45 

1.57 

Prepare  for  and  conduct  Sergeant's  Time  and  Opportunity  training. 

3.36 

1.12 

Prepare  a  training  outline/lesson  plan. 

Leadership 

4.64 

0.67 

Identify  the  leadership  duties,  responsibilities,  authority,  and 
requirements  of  officers  and  NCOs. 

4.00 

1.08 

Know  the  policies  and  procedures  of  the  chain  of  command  and  of 
the  NCO  support  channel. 

4.00 

1.08 

Identify  the  principles  of  BE,  KNOW ,  DO. 

3.92 

1.04 

Know  the  principles  of  discipline. 

3.85 

0.99 

Identify  the  risk  management  process. 

3.77 

1.17 

Identify  the  steps  of  assuming  a  leadership  position. 

3.69 

0.95 

Know  of  the  NCO  Education  System  (NCOES). 

3.69 

1.25 

Know  the  characteristics  of  effective  counseling. 

3.69 

1.32 

Understand  the  principles  of  the  rules  of  engagement  (ROE)  and  use 
of  force. 

3.62 

1.19 

Know  the  steps  in  troop  leading  procedures  (TLP). 

3.46 

1.13 

Prepare  for  &  conduct  inspections  and  on-the-spot  corrections. 

3.46 

1.51 

Resolve  an  ethical  problem. 

3.31 

1.49 

Know  the  problem  solving  steps  in  decision  making. 

2.92 

0.86 

Understand  the  requirements  of  the  semi-centralized  (E5-E6) 
promotion  systems. 

2.38 

0.77 

Know  the  principles  of  financial  management. 

1.77 

0.83 

Identify  the  Army's  homosexual  conduct  policy. 
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Appendix  C 

Mean  Weights  and  Standard  Deviations  for  Common  Task  Areas 


Mean1 

SD2 

SL1  Subject  Areas 

17.42 

7.42 

First  Aid 

15.42 

7.97 

M16  Rifle  /  M4  Carbine  /  M9  Pistol 

8.33 

4.58 

Nuclear,  Biological,  &  Chemical  (NBC) 

8.17 

3.04 

Communicate:  Radio  &  Telephone 

7.92 

3.90 

Combat  Techniques  (Survive) 

7.92 

4.17 

Navigate,  Mounted  &  Dismounted 

5.92 

2.87 

Defense  Measures:  Camouflage,  SALUTE,  OPSEC 

5.67 

3.37 

M60  Machine  Gun  /  M249  SAW  /  M240B 

4.08 

2.07 

Hand  Grenades  &  Land  Mines  (M18A1  Claymore) 

4.08 

1.24 

Individual  Conduct  &  Laws  of  War 

3.17 

1.59 

Remains  Reporting  &  Handling 

3.08 

1.78 

Caliber  .50  M2  Machine  Gun 

3.00 

1.60 

M203  mm  Grenade  Launcher  (Ml 6  Series) 

2.92 

1.73 

Ml 36  Launcher  AT4  Light  Anti-Tank  Weapon 

2.92 

1.62 

MK19  40mm  Grenade  Launcher  Machine  Gun 

SL2  Subject  Areas 

17.58 

8.37 

Combat  Techniques  (Survive) 

17.50 

5.78 

First  Aid:  MEDEVAC,  Preventive  Medicine 

16.42 

4.60 

Equipment  Checks:  PMCS,  Supply  Discipline,  Property  Accountability 

14.08 

3.73 

Defense  Measures:  Squad  Defense 

12.25 

5.17 

Navigate:  Map  Overlays 

11.25 

5.12 

Risk  Management:  Accident  Prevention 

10.92 

6.26 

Nuclear,  Biological,  &  Chemical  (NBC) 

1  Reflects  the  average  ranking  of  individuals  who  distributed  100  points  among  the  areas  when  asked  how  many 
points  they  would  give  each  area  for  importance  in  testing.  The  number  equals  a  percentage  assignment  and  the  total 
is  100% 

2  Standard  Deviation.  This  reflects  how  much  raters  differed  individually  from  the  reported  mean  rating.  Low  SD 
numbers  (approaching  zero)  indicate  raters  gave  the  same  or  close  to  the  same  rating.  Higher  SD  numbers  indicate 
individual  raters  differed  more  on  their  ratings. 
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Appendix  D 

Means  and  Standard  Deviations  of  Rankings  of  Tasks  within  Subject  Areas 

for  SL1  and  SL2  Common  Tasks 


Mean1 

SD2 

Skill  Level  1 

First  Aid 

3.25 

4.81 

Evaluate  a  casualty 

4.83 

4.09 

Perform  Mouth-to-Mouth  Resuscitation 

5.33 

2.74 

Perform  First  Aid  for  Bleeding  of  an  Extremity 

6.67 

3.89 

Perform  First  Aid  to  Clear  an  Object  Stuck  in  the  Throat  of  a  Conscious  Casualty 

6.67 

3.92 

Perform  First  Aid  for  Heat  Injuries 

6.92 

3.80 

Perform  First  Aid  for  an  Open  Abdominal  Wound 

7.25 

3.57 

Perform  First  Aid  for  an  Open  Chest  Wound 

7.25 

2.67 

Perform  First  Aid  for  an  Open  Head  Wound 

7.83 

3.61 

Perform  First  Aid  to  Prevent  or  Control  Shock 

9.25 

2.30 

Perform  First  Aid  for  Bums 

9.92 

3.48 

Perform  First  Aid  for  a  Suspected  Fracture 

10.00 

5.17 

Practice  Individual  Preventive  Medicine  Countermeasures 

10.75 

3.39 

Perform  First  Aid  for  Nerve  Agent  Injury 

10.75 

3.31 

Perform  First  Aid  for  Cold  Injuries 

13.25 

2.80 

Transport  a  Casualty 

Communicate:  Radio  &  Telephone 

1.92 

1.00 

Perform  Voice  Communications 

2.00 

0.60 

Communicate  Via  a  Tactical  Radio  in  a  Secure  Net 

2.08 

0.90 

Communicate  Via  a  Tactical  Telephone 

M16  Rifle  IM4  Carbine /M9  Pistol 

1.58 

0.90 

Maintain  weapon 

3.33 

1.30 

Load  an  M4  or  M9 

3.50 

1.57 

Engage  targets  with  an  M16-Series  Rifle  Using  a  Night  Vision  Sight  AN/PVS-4 

3.92 

1.73 

Engage  targets  with  an  M16-Series  Rifle  Using  an  AN/PAS-13  Series  Thermal  Sight 

4.08 

1.51 

Engage  targets  with  an  M4  or  M9 

4.58 

1.73 

Unload  an  M4  or  M9 

Nuclear,  Biological,  &  Chemical  (NBC) 

2.17 

1.95 

Maintain  Your  Assigned  Protective  Mask 

2.33 

1.07 

Protect  Yourself  from  Chemical/Biological  Contamination  Using  Your  Assigned  Mask 

1  Reflects  the  average  ranking  of  the  task  within  the  subject  area.  Respondents  picked  the  highest  priority  task  as  #1, 
second  highest  #2,  etc.,  rating  every  task  listed.  Subject  areas  with  only  one  task  were  not  rated. 

2  Standard  Deviation.  This  reflects  how  much  raters  differed  individually  from  the  reported  mean  rating.  Low  SD 
numbers  (approaching  zero)  indicate  raters  gave  the  same  or  close  to  the  same  rating.  Higher  SD  numbers  indicate 
individual  raters  differed  more  on  their  ratings. 
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Means  and  standard  deviations  of  rankings  of  tasks  within  subject  areas  for  SL1  and  SL2 
Common  Tasks  ( continued ) 


3.50 

1.45 

React  to  Chemical  or  Biological  Hazard/ Attack 

3.75 

1.42 

Protect  Yourself  from  NBC  Injury/Contamination  with  MOPP  Gear 

5.00 

1.76 

Decontaminate  Yourself  &  Individual  Equipment  Using  Chemical  Decon  Kits 

5.92 

2.19 

React  to  Nuclear  Hazard/Attack 

6.08 

1.08 

Detect  Chemical  Agents  Using  M8  or  M9  Detector  Paper 

7.25 

1.22 

Respond  to  Depleted  Uranium 

Combat  Techniques  (Survive) 

3.92 

2.19 

React  to  Direct  Fire  While  Mounted 

4.42 

3.18 

Move  Under  Direct  Fire 

5.25 

3.36 

Select  Temporary  Fighting  Positions 

5.42 

2.68 

React  to  Indirect  Fire  While  Dismounted 

6.00 

2.59 

React  to  Indirect  Fire  While  Mounted 

6.33 

3.82 

Perform  Duty  as  a  Guard 

6.92 

2.81 

Mover  Over,  Through,  or  Around  Obstacles  (Except  Minefields) 

7.08 

4.21 

Operate  a  Vehicle  in  a  Convoy 

7.42 

3.12 

Construct  Individual  Fighting  Positions 

7.58 

3.15 

React  to  Unexploded  Ordnance  Hazards 

7.83 

4.26 

Practice  Noise,  Light,  &  Litter  Discipline 

9.83 

2.41 

Plan  Use  of  Night  Vision  Devices 

Defense  Measures 

2.50 

1.00 

Implement  Operations  Security  (OPSEC)  Measures 

2.58 

1.68 

Control  Entry  to  &  Exit  from  a  Restricted  Area 

3.08 

1.24 

Report  Intelligence  Information 

3.17 

1.53 

Camouflage  Yourself  &  Your  Individual  Equipment 

3.67 

1.50 

Camouflage  Equipment 

Hand  Grenades  &  Land  Mines  (M18A1  Claymore) 

1.92 

1.08 

Perform  Safety  Checks  on  Hand  Grenades 

2.50 

0.90 

Employ  Hand  Grenades 

2.75 

0.97 

Employ  an  M18A1  Claymore  Mine 

2.83 

1.40 

Locate  Mines  by  Probing 

Navigate,  Mounted  &  Dismounted 

1.50 

0.52 

Navigate  from  One  Point  on  the  Ground  to  Another  Point  While  Dismounted 

1.50 

0.52 

Navigate  from  One  Point  on  the  Ground  to  Another  Point  While  Mounted 

M60  Machine  Gun  / M249  SAW / M240B 

1.33 

0.65 

Maintain  Weapon 

2.83 

0.72 

Load  M60  or  M240B 
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Means  and  standard  deviations  of  rankings  of  tasks  within  subject  areas  for  SL1  and  SL2 
Common  Tasks  (continued) 


3.17 

1.47 

3.75 

1.22 

3.92 

1.31 

1.08 

0.29 

1.92 

0.29 

1.50 

0.80 

2.50 

1.00 

3.33 

1.44 

3.75 

1.06 

3.92 

1.31 

1.67 

1.15 

2.42 

1.08 

3.33 

1.56 

3.50 

0.90 

4.08 

1.08 

1.58 

0.79 

2.17 

0.72 

2.25 

0.87 

1.67 

1.23 

2.67 

0.89 

3.00 

1.21 

3.50 

1.31 

4.17 

1.27 

1.58 

1.00 

2.33 

0.65 

3.00 

1.13 

4.42 

1.62 

5.08 

1.38 

5.08 

0.51 

Engage  Targets  with  Weapon 
Unload  M60  or  M240B 
Prepare  a  Range  Card  for  Weapon 

M136  Launcher  AT4  Light  Anti-Tank  Weapon 

Prepare  an  M136  Launcher  for  Firing 

Restore  an  Ml 36  Launcher  for  Carrying  Configuration 

Caliber  .50  M2  Machine  Gun 

Maintain  a  Caliber  .50  M2  Machine  Gun 

Load  a  Caliber  .50  M2  Machine  Gun 

Engage  Targets  with  a  Caliber  .50  M2  Machine  Gun 

Unload  a  Caliber  .50  M2  Machine  Gun 

Prepare  a  Range  Card  for  a  Caliber  .50  M2  Machine  Gun 

M203  40  mm  Grenade  Launcher  (M16  Series) 

Maintain  an  M203  Grenade  Launcher 
Load  an  M203  Grenade  Launcher 
Engage  Targets  with  an  M203  Grenade  Launcher 
Correct  Malfunctions  of  an  M203  Grenade  Launcher 
Unload  an  M203  Grenade  Launcher 

Remains  Reporting  &  Handling 
Recover  Isolated  Remains 
Evacuate  Isolated  Remains 

Inter  Isolate  Remains  (After  Receiving  Authorization) 

MK19  40mm  Grenade  Launcher  Machine  Gun 

Maintain  an  MK19  Machine  Gun 

Perform  a  Function  Check  on  an  MK19  Machine  Gun 

Load  an  MK19  Machine  Gun 

Engage  Targets  with  an  MK19  Machine  Gun 

Prepare  a  Range  Card  for  an  MK19  Machine  Gun 

Individual  Conduct  &  Laws  of  War 

Comply  with  the  Uniform  Code  of  Military  Justice  (UCMJ) 

Comply  with  the  Law  of  War  &  the  Geneva  &  Hague  Conventions 
Comply  with  the  Requirements  of  the  Code  of  Conduct 
Comply  with  the  Army's  Equal  Opportunity  &  Sexual  Harassment  Policies 
Interact  with  News  Media 

Support  Unit  &  Family  Readiness  Through  the  Army  Family  Team  Building  (AFTB) 
Program 
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Means  and  standard  deviations  of  rankings  of  tasks  within  subject  areas  for  SL1  and  SL2 
Common  Tasks  ( continued j 

Skill  Level  2 
First  Aid 


1.83 

0.83 

Request  Medical  Evacuation 

2.00 

0.74 

Evacuate  Casualties 

2.17 

0.94 

Implement  Preventive  Medicine  Measures  (PMM) 

Nuclear ,  Biological,  &  Chemical  (NBC) 

3.33 

2.15 

Protect  Yourself  from  NBC  Injury/Contamination  When  Changing  MOPP  Gear 

5.67 

2.53 

Identify  Chemical  Agents  Using  M256-Series  Chemical  Agent  Detector  Kit 

9.58 

1.98 

Measure  Radiation  Dose  Rate  &  Total  Dose 

5.58 

2.84 

Conduct  Unmasking  Procedures 

6.58 

2.43 

Supervise  the  Crossing  of  a  Contaminated  Area 

6.58 

2.61 

Submit  an  NBC  1  Report 

2.58 

1.73 

Implement  MOPP 

9.25 

1.42 

Supervise  the  Employment  of  NBC  Markers 

6.17 

2.12 

Supervise  Unit  Preparation  for  an  NBC  Attack 

7.50 

2.50 

Report  NBC  Information  Using  NBC  4  Report 

3.17 

3.19 

Conduct  a  Mask  Fit  Test 

Survive  (Combat  Techniques) 

2.58 

1.31 

Adjust  Indirect  Fire 

3.50 

0.52 

Use  Visual  Signaling  Techniques 

2.08 

1.16 

Supervise  Construction  of  a  Fighting  Position 

1.83 

0.58 

Establish  an  Observation  Post 

Equipment  Checks 

2.00 

1.13 

Supervise  PMCS 

3.92 

1.24 

Enforce  Compliance  with  Supply  Discipline  Procedures 

3.83 

0.72 

Enforce  Property  Accountability  Policies 

3.92 

0.79 

Enforce  Compliance  with  Property  Accountability  Policies 

1.33 

0.49 

Inspect  Equipment  for  Accountability,  Cleanliness,  &  Serviceability 

Navigate 

N/A 

Use  a  Map  Overlay 

Risk  Management 

N/A 

Employ  Accident  Prevention  Measures  &  Risk  Management  Process 

Defense  Measures 

N/A 

Conduct  a  Defense  by  a  Squad 
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